Skip to content

feat: support codex custom tool type#101

Draft
haoshan98 wants to merge 6 commits into
vllm-project:mainfrom
EmbeddedLLM:codex-custom-tool
Draft

feat: support codex custom tool type#101
haoshan98 wants to merge 6 commits into
vllm-project:mainfrom
EmbeddedLLM:codex-custom-tool

Conversation

@haoshan98

@haoshan98 haoshan98 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Codex declares freeform tools such as apply_patch with type: "custom". Unlike function tools, these declarations can
include an opaque grammar, and their calls carry raw text in input rather than JSON in arguments. The typed Responses
path previously could not preserve that protocol through upstream inference, streaming, storage, and continuation.

  • Add typed request, response, and input models for custom tool declarations, custom_tool_call items, and custom_tool_call_output results while preserving unknown format fields.
  • Forward custom tool declarations to the upstream Responses endpoint without converting them to function tools.
  • Normalize and accumulate custom-tool streaming input events, return custom calls to the client for execution, and retain the call/output pair during stateful continuation.
  • Keep custom calls client-owned in the shared tool loop while gateway-owned MCP and web-search calls continue to execute through their registered handlers.
  • Persist terminal WebSocket responses before emitting completion so Codex can immediately continue a client-owned tool call without losing stored state when it closes the socket.
  • Handle Codex WebSocket generate: false startup prewarm requests locally, persist their prompt and tool context, and avoid unnecessary upstream inference.
  • Preserve and log structured upstream streaming failure details to make rejected tool requests diagnosable.
  • Add an isolated catalog/cache seed helper that writes both Codex catalog files so freeform apply_patch remains
    available after the ordinary model cache expires.
  • Extend the cassette recorder with custom-call continuation support and add direct-vLLM HTTP, gateway HTTP/SSE,
    gateway WebSocket, OpenAI HTTPS/SSE, and OpenAI WebSocket recordings that preserve raw input and
    custom_tool_call_output.
  • Document direct gateway, codex exec, and interactive Codex verification workflows.

Signed-off-by: haoshan98 <haoshanw@gmail.com>
Signed-off-by: haoshan98 <haoshanw@gmail.com>
Signed-off-by: haoshan98 <haoshanw@gmail.com>
Signed-off-by: haoshan98 <haoshanw@gmail.com>
Signed-off-by: haoshan98 <haoshanw@gmail.com>
Signed-off-by: haoshan98 <haoshanw@gmail.com>

# Conflicts:
#	crates/agentic-server/src/handler/websocket/responses.rs
#	crates/agentic-server/tests/responses_websocket_test.rs
};
Ok(Self {
id,
status: Some("in_progress".to_owned()),

@maralbahari maralbahari Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can use MessageStatus enum.

/// Skipped when empty so vLLM does not receive an empty array.
#[serde(skip_serializing_if = "Option::is_none")]
pub tools: Option<Vec<FunctionTool>>,
pub tools: Option<Vec<Value>>,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this type change from FunctionTool to Value seems like messing up most of the current tool contract. revisit this I think even the free form should be normalized to FunctionTool type

item.input = input;
}
if item.status.as_deref().is_none_or(|status| status == "in_progress") {
item.status = Some("completed".to_owned());

@maralbahari maralbahari Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of using the hardcoded string for status. you can use the MessageStatus enum. see the above inflight items how they use it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants